Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning

In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, especially for deep RL domains, where a state is represented by high-dimensional image frames. In this paper, we propose a novel informed exploration framework for deep RL, where we build the capability for an RL agent to predict over the future transitions and evaluate the frequentness for the predicted future frames in a meaningful manner. To this end, we train a deep prediction model to predict future frames given a state-action pair, and a convolutional autoencoder model to hash over the seen frames. In addition, to utilize the counts derived from the seen frames to evaluate the frequentness for the predicted frames, we tackle the challenge of matching the predicted future frames and their corresponding seen frames at the latent feature level. In this way, we derive a reliable metric for evaluating the novelty of the future direction pointed by each action, and hence inform the agent to explore the least frequent one

MS

thesisThe purpose of the current study was to nvestigate the effects of developmental massage therapy (DMT) during early postnatal life on growth, body composition, and skeletal development in juvenile and young adult rats. Twenty-four rat pups received a daily session of DMT (10 minutes/day) from D6 to D10 of postnatal life and were compared to matched controls (CTL, n=24). Body composition, soft tissue lean mass, fat mass, bone mineral density (BMD), bone mineral content (BMC), and bone area (BA) were measured by peripheral dual energy X-ray absorptiometry (pDXA); bone strength (Peak Load) and intrinsic stiffness on femur shaft were tested by three-point bending (MTS); cortical and cancellous bone histomorphometric measurements were performed on the tibia shaft and metaphysis. Food and water intake were monitored and did not differ among groups between weaning (D21) and D60. Body weight, body length and soft tissue lean mass at D21 were significantly greater in the DMT cohort; at D60 soft tissue lean mass was still greater in DMT groups but no differences were detected in body weight and body length. Moderate to strong correlations were found between body weight and soft tissue lean mass and measured bone outcome variables. BMD by DXA was significantly higher in D21 DMT rats and BA was only greater in D60 DMT males. Femur diameter and length were significantly greater in D60 DMT males. Bone strength and the endosteal percent mineral surface were significantly increased in D60 DMT females compared to CTL. In addition, DMT treatment improved the cancellous bone mineral surface, mineral apposition rate and bone formation rate on the trabecular bone surface area in D21 female rats. Treatment and sex interactions were found for BMC and BA at D60. In summary, DMT during early postnatal life elicited anabolic effects on muscle-bone relationships in juvenile (D21) and young adult (D60) rats. These findings were confirmed histologically by increased percent mineral surface and mineral apposition rates. It is speculated that the positive skeletal outcomes were, in part, due to a modulation of stress by DMT resulting in an improved balance of the neuroendocrine system and regulation of the muscle-bone unit. The absence of biochemical markers as well as body composition and bone studies in older animals suggest a need for future studies to verify whether early life DMT has a prolonged positive impact on lifelong skeletal health

Homoclinic orbits for periodic second order Hamiltonian systems with superlinear terms

We obtain the existence of nontrivial homoclinic orbits for nonautonomous second order Hamiltonian systems by using critical point theory under some different superlinear conditions from those previously used in Hamiltonian systems. In particular, an example is given to illustrate our result

The new glucose revolution: is the authoritative guide to the glycemic index the right dietary solution for lifelong health?

Journal ArticleThe persistence of an epidemic of obesity, coronary heart disease and type 2 diabetes suggests that new nutritional strategies are needed if the epidemic is to be overcome. A promising nutritional approach suggested by this thematic review is carbohydrate restriction. Under conditions of carbohydrate restriction, fuel sources shift from glucose and fatty acids to fatty acids and ketones, and that ad libitum?fed carbohydrate-restricted diets lead to appetite reduction, weight loss, and improvement in surrogate markers of cardiovascular disease. Recent researches focus on the debate among professionals regarding the use of the glycemic index (GI) for meal planning. However, evidence from individual trials about benefits and risks of these diets to achieve weight loss and modify cardiovascular risk factors is preliminary. In epidemic studies, there is limited evidence that a low GI diet is beneficial for a reduced risk of developing diabetes or prevalence of insulin resistance, weight loss or satiety, and other cancers. The GI can be used as an adjunct for the fine tuning of postprandial blood glucose responses, particularly in diabetic patients. Other food/meal-planning interventions have been shown to be more effective than the use of the GI

Test for Spatio-Temporal Counts Being Poisson

The new Log-Linear Test (TL) is proposed to identify when the Poisson model fails for a collection of count random variables. TL is shown to have better rejection rate with small sample size and essentially the same power compared to a classical Fisher-Bohning’s Statistic TF for standard alternatives to Poisson